Three-dimensional (3D) maps provide a user with a representation of a physical environment. 3D maps represent internal environments (e.g., indoors) or external environments (e.g., outdoors). 3D maps may be generated using specialized optical display devices. In some cases, such devices are mounted on or around the user's head. In other cases, such devices may be hand-held devices such as smartphones. 3D maps have various applications. For example, a 3D map may be used in video game systems, augmented reality systems, etc.